METHOD AND APPARATUS FOR ADAPTIVE PIXEL CORRECTION OF A 
MULTI-COLOR MATRIX 



BACKGROUND OF THE INVENTION 

The priority benefit of the October 10, 2002 filing dates of U.S. provisional patent 
applications serial nos. 60/417,152 and 60/417,142 are hereby claimed. 



5 Field of the Invention 

The present invention relates to a method and apparatus for adaptive pixel defect 
correction in a multi-color matrix based sensor arrangement through the assessing of edge 
continuity, a computer program and a computer program for controlling such method in a 
computer, and an image sensing facility encompassing such apparatus. In particular, the 
10 present invention relates to a method for adaptive pixel defect correction in a multi-color 
matrix-based sensor arrangement that assesses pixel values of a two-dimensional multi-pixel 
kernel around a particular pixel when determining an apparent defect value of said particular 
pixel. 



15 Description Of Related Art 

Multi-color and matrix-based sensor arrangements are coming into rapidly expanding 
use, where the matrix size expressed in the number of pixels is increasing more or less 
exponentially in time, and the pixel size is generally decreasing likewise. The probability that 
a particular pixel will be defective is decreasing as well, but not at the same high rate, so that 



the overall probability for a matrix to have a defective pixel is on the rise. In many cases, such 
is commercially unacceptable, so there exists a growing need to provide for an algorithmic 
procedure that will correct as much as possible the effect of such defective pixels, while 
requiring only minimal memory facilities and power consumption, needing to access only a 
5 small number of image lines/columns at a time, and being applicable to various different 
defect categories whilst being independent of other DSP (= Digital Signal Processing) -based 
algorithms, such as those for removing false colors, overall influencing of hue or luminance, 
or others. The fields of application of such CCD (= Charged Coupled Device), CMOS (= 
Complementary Metal Oxide Semiconductor), and similarly organized sensors are digital still 

10 cameras, digital cinematography, video cameras, scientific imaging, broadcast cameras, 
surveillance, security monitoring, and various others. 

By itself, United States Patent Publication 2002/00151 1 1 to Harada and published on 
February 7, 2002 discloses an image processing apparatus and associated method. In 
contradistinction with the present invention, Harada uses a two-dimensional adaptive 

15 interpolation wherein a hard decision is made to select the pixels that will be averaged for 
finding the defective pixel. Moreover, no correlations across different color planes are used, 
so that the reference is based on a different perceptual model from the present invention. 

SUMMARY OF THE INVENTION 

20 In consequence, amongst other things, it is an object of the present invention to 

introduce the cross-correlation between different color planes as a perceptually-based 
mechanism for correcting defective pixels. 

This an other objects are achieved in a method of adaptive pixel correction of a multi- 
color matrix from a sensor that includes collecting pixel values, generating plural vectors, 

25 normalizing values and determining an estimated value. The collecting pixel values collects 
values of a two-dimensional multi-pixel kernel around a particular pixel from the multi-color 
matrix. The generating plural vectors generates vectors from the kernel where each vector 
has the particular pixel as a central element. The normalizing values normalizes values for 
nearest neighbor elements to the central element for each vector. The deterniining an 

30 estimated value determines the value based on the normalized values. 
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This an other objects are also achieved in a system for adaptive pixel correction of a 
multi-color matrix from a sensor comprising a processor and first, second, third and fourth 
modules. The first module controls the processor to collect pixel values of a two-dimensional 
multi-pixel kernel around a particular pixel from the multi-color matrix. The second module 
5 controls the processor to generate plural vectors from the kernel where each vector has the 
particular pixel as a central element. The third module controls the processor to normalizing 
values for nearest neighbor elements to the central element for each vector. The fourth 
module controls the processor to determine an estimated value based on the normalized 
values. 

10 This an other objects are also achieved in a computer readable medium to control a 

processor for adaptive pixel correction of a multi-color matrix from a sensor. The medium 
includes first, second, third and fourth modules. The first module controls the processor to 
collect pixel values of a two-dimensional multi-pixel kernel around a particular pixel from the 
multi-color matrix. The second module controls the processor to generate plural vectors from 

15 the kernel where each vector has the particular pixel as a central element. The third module 
controls the processor to normalizing values for nearest neighbor elements to the central 
element for each vector. The fourth module controls the processor to determine an estimated 
value based on the normalized values. 



20 BRIEF DESCRIPTION OF DRAWINGS 

The invention will be described in detail in the following description of preferred 
embodiments with reference to the following figures wherein: 
FIG. 1 , a Bayer matrix of image points; 

FIG. 2, the sequential arrangement of the image points for processing; 
25 FIGS. 3a-3c, the pixel-based algorithm processing steps; 

FIG. 4, a flow diagram of a preferred algorithm; 

FIG. 5, a diagram showing the reason for t aliasing tolerance in the correction error 

test; 

FIGS. 6a, 6b, the creation of limits for the error correction test; 
30 FIGS. 7a, 7b, examples of the error correction test; 

FIG. 8, a flow diagram of a DSP pipeline with error correction test. 
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DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

A co-pending patent application by the present inventor, titled "Method For 
Reconstruction Of Pixel Color Values," identified by Attorney Docket Number 12547 and 
5 assigned to the same assignee as the present application, relates to object-based color 
reconstruction and is incorporated herein by reference. The reconstruction of color pixel 
values is based on soft decisions in the various color planes, and in particular, by taking into 
account the spatial response of the human visual system. The results of the present invention 
may be applied as input to the co-pending application's invention, but such represents no 
10 express restriction. 

The present invention minimizes perceptual error, utilizes pictorial properties to make 
intelligent solutions when correcting defective points, and can correct a wide range of shapes 
and sizes of a defect. Such defects may have the form of single pixels or small clusters. For 
situations as CCD, the defects may include a whole column or a relatively substantial part 
15 thereof. These defects have relatively high spatial frequencies and will often be prominently 
visible to a user or observer and therefore be unacceptable. 

The present invention is, inter alia, based on recognizing the perceptual relevance of a 
user's expectation that edges which are present in the sensed image will generally be 
continuous, and that disturbance or interruption of such edges will often be due to 
20 disturbances of the sensor per se. 

The property of pictorial image R-G-B correlation can be demonstrated by taking the 
gradient of each R, G, B image component and by analyzing where the gradients differ. In an 
input image a[m,n,X] that is a function of spatial m (horizontal), n (vertical) and R-G-B 
components (X), the overall gradient may be defined as 
25 Va = m da/dx 6+ n da/dy + X da/dz. 

where m, n and X are unit vectors. 

In a given pictorial image, it generally holds that the R-G-B values are correlated 
within objects and along edges. This means that the rates of change of R, G, and B are 
substantially constant for a point in regions with low spatial frequency and also substantially 
30 constant along the edge direction in regions with high spatial frequency. Now ideally, one 
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would like to find the edge direction and utilize the two neighboring pixels in that direction to 
estimate the defect pixel a[mO,nO). However, in a Bayer matrix arrangement each non-Green 
pixel is exclusively surrounded by pixels of different colors. For Green pixels, four diagonal 
neighbors are Green indeed, but the other four neighbors are still non-Green. However, the 
5 R-G-B colors of an image are locally correlated to each other even across the color 
differences. Therefore, although neighboring pixels may sample different colors, according to 
the invention they are still correlated to the defect pixel's color plane. 

The cross-color approach represents a particularly advantageous refinement. The 
above is exploited by calculating the gradient of the non-defect color plane on each side for 

1 0 each edge direction and by applying this gradient to the nearest known pixel of the same color 
as the defect on each side of the defect for each edge direction. We then correlate the nearest 
pixels to the defect pixel's color and acquire the nearest known data. Finally, we may select 
the edge direction by merely determining which two neighboring pixels of a given edge 
direction have the least difference in magnitude, once correlated to the defect pixel's color. In 

15 a relatively simplistic implementation, the defect value may be calculated as the arithmetic 
mean of those two pixels. However, a fuller realization of the invention entails to calculate 
the weighted sum of all neighboring pixels with their weights based on their alignment with 
the true edge direction. 

In an embodiment of the invention, a method exploits this cross-color approach to 

20 interpolating a defective pixel. The invention also relates to a computer program having 
program instructions for controlling a computer to implement a method that exploits this 
cross-color approach to interpolating a defective pixel, and to such computer program realized 
as a computer readable medium represented within a tangible read-only computer memory 
medium or being carried by an electrical signal. The invention also relates to an apparatus 

25 arranged for implementing a method that exploits this cross-color approach to interpolating a 
defective pixel and to an image sensing facility comprising such an apparatus. Further 
advantageous aspects of the invention are discussed herein. 

FIG. 1 illustrates a Bayer matrix of image points for use with the present invention. 
Alternatively, other arrangements could be used, such as through using subtractive colors, 

30 such as cyan, magenta, and yellow, or through various other published embodiments. As 
shown, the matrix has Green pixels arranged according to a chessboard organization. The 



remaining pixels have been used for Red pixels in odd rows and Blue pixels in even rows. 
The dashed square for example corresponds with a Blue pixel 

FIG. 2 illustrates the sequential arrangement of the image points for processing. 
Along a row, they are presented in sequence, followed by the next row presented in the same 
5 sequential order. It would be clear from this Figure, as to how the pixels pertaining to a 
particular kernel shape would be organized for use in the processing hereinafter. 

Hereinafter, the principles of the preferred embodiment will be discussed. The 
sampled input image a[m,n] of M columns and N rows in total lets each image point in 
sequence be tested against the defect map. Finding a defect will initiate the correction 

10 thereof. The defect proper will often be signaled by a defect map detection or a defect 
detection apparatus. Such is state of the art. 

Subsequently to the finding of a defect map, the algorithm of adaptive pixel correction 
of a multi-color matrix from a sensor includes the following. 

Pixel values from the multi-color matrix a[m,n] are input to the algorithm column by 

15 column within a row and then advancing one row after the last column is input, and the 
column by column input for the row is repeated (see FIG. 2). In the example discussed 
herein, 7 rows are buffered locally in order to process a 7 x 7 matrix kernel. A particular 
pixel, designated to be at mo, no , is a pixel 4 columns behind and 4 rows behind the current 
pixel input. This particular pixel is the pixel under attention. Pixel values of a two- 

20 dimensional multi-pixel kernel around the particular pixel are extracted from the multi-color 
matrix to form a 7 x 7 matrix kernel with the particular pixel being central to the kernel. The 
intended color of the central pixel is assessed. The eight nearest neighbor pixels to the 
particular pixel form parts of four directional "vectors." Each directional vector includes 7 
pixels disposed in a straight line that passes through one nearest neighbor pixel, then through 

25 the particular pixel, and then through another nearest neighbor pixel. 

In the 7 x 7 matrix kernel, the four directional "vectors" are derived at respective 
angles that differ by 45°, and which vectors are centered at the particular pixel. The central 
pixel is under attention, is located at mo, and has a value of a[ mo, ]. The four vectors 
have seven elements each, where each element is indexed by nn which may vary over a 

30 range from -3 to +3. The directional four vectors are extracted from the 7 x 7 matrix kernel 



so that the vector element index, denoted nn, in each vector is 0 at mo, no and the value of 
all vectors at nn = 0 is 

dj[ nn = 0 ] = a[ m 0 , n 0 ] 

for all vectors, i = 1 to 4. The four vectors are defined by: 

5 

di [ nn = -3 to +3 ]={ a [ m 0 , n_ 3 J...a [ m 0 , n 0 ]...a [ m 0 , n +3 ]}; 

d 2 [ nn = -3 to +3 ]={ a [ m_ 3 , n 3 ]...a [ m 0 , n 0 ]...a [ m +3 , n. 3 ]}; 
d 3 [ nn = -3 to +3 ]={ a [ m_ 3 , n 0 ]...a[ m 0 , n 0 J ...a [ m +3 , n 0 J}; and 
d 4 [ nn = -3 to +3 ]={ a [ m. 3 , n. 3 ]...a [ m 0 , n 0 ]...a [ m +3 , n +3 ]}. 

10 

If the defect map indicates a column is defective at the central image point, the 
vertical vector in question is thereupon ignored, thus reducing the number of vectors to 
process from four to three. This generally is only relevant for a CCD sensor. 

Next, all active vectors are checked for uncorrected error element. Uncorrected error 

15 elements in vectors are elements corresponding to pixels in the multi-color matrix from the 
sensor that are marked as defective. Because all defects are corrected in a column by column 
then row by row sequence, defects in pixels that have been processed previously should have 
been corrected already, but defects in pixels that have yet to be processed may have an 
uncorrected error. Such uncorrected error pixels in the kernel (elements in the vectors) are in 

20 addition to the particular pixel under attention (i.e., the defect point herein) that is located at 
nio, Ho . For example, from the 7 x 7 matrix kernel surrounding the particular pixel under 
attention, the vector di [ nn = -3 to +3 ] is extracted, and this directional vector has values 
{ a[ mo, n_ 3 ]..a[ mo, no ]..a[ mo, n +3 ]} . The values of the vector elements for the index 
range nn = -3 to -1 have already been processed (see FIG. 2), the value for the vector 

25 element for index nn = 0 is the particular pixel under attention (i.e., the defect point herein), 
and the values for the vector elements for the index range nn = +1 to +3 have yet to be 
determined and could still have uncorrected errors at an index that may vary from nn defect 1 
to 3. If a defect exists in one of the vectors (i.e., in addition to a defect at nn = 0), then the 
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element corresponding to the defect will be interpolated locally within the vector on the same 
side of the central element (i.e., the same side of nn = 0). The interpolation depends on the 
vector to ensure that a step function on one side of the vector will not be averaged into the 
other side. The vector itself will therefore not be later falsely considered as the edge itself. A 
5 vector with an uncorrected error element is processes as follows: 

if midfeet = +3, dj [ nn defect ] = d } [ nn de fect - 2 ] , 
if nn defect = +1, dj [ nn def ect ] = dj [ nn defect + 2 ] , and 
otherwise, d s [ nn de fect ] = dj [ -nn de fect ] • 

10 

In this way the uncorrected error element at nn defect is "filled in" with a value so that a value 
can be estimated for the particular pixel under attention (i.e., the defect point herein) that is 
located at mo, no . 

Next, vector elements corresponding to pixels nearest to the pixel under attention at 
15 mo, no (i.e., the values at dj[nn = -1] and dj[ nn = +1]) are normalized to the color plane of 
the defect. Normalization is accomplished by calculating the gradient of the non-defect color 
plane on each side of d; [ 0 ] and by applying the gradient to the nearest sampled elements dj 
[-2 ] and ds [ +2 ] of the same color plane as the particular pixel under attention. The 
procedure is carried out as follows for each vector dj. 
20 The gradients of the non-defect color plane on either side of the particular pixel are 

calculated for each vector according to: 

V i [-l] = {d i [-l]-d i [-3]}/2,and 
V i [+l] = {d i [+l]-d i [+3]}/2. 

25 

where Vj [-1] and Vj [+1] are the single-sided single-color gradients on the minus and plus 
side of the particular pixel under attention, respectively. 

Then theses gradients from the non-defect color plane are correlated to the pixels in 
the defect color plane that are nearest to the particular pixel under attention. This is 
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accomplished by defining new vector elements to estimate the nearest sample elements in the 
defect color plane to the particular pixel under attention. Let the normalized 6 vector 
elements be given by: 



5 d i [-l] = d i [-2]+V i [-l],and 

di [ +1 1 = di [ +2 ] + Vi [ +1 1. 

where di [ -1 ] and d{ [ +1 ] are the single-sided normalized vector elements on the minus 
and plus side of the particular pixel under attention, respectively. 
1 0 Then, for vectors for which normalized vector elements were found, we substitute the 

normalized vector elements for the values at di[nn = -1] and dj[ nn = +1]) in the color plane 
of the defect as follows: 

dj[-l]=di[-l ],andd i [+ll=d i [+l J. 

15 

The nearest sampled elements of the same color plane as the particular pixel under 
attention are sample measurement found at d, [-2 ] and dj [ +2 ] . The single-sided normalized 
vector elements dj [ -1 ] and dj [ +1 ] are estimates of samples of the same color as the 
particular pixel under attention that could have been made at dj[nn = -1] and dj[ nn = +1] if 
20 such sample measurements were made. In this way, the values of the directional vector 
elements from dj [-2 ] to dj [ +2 ] , omitting only di [0] , have been either sampled or 
estimated, and the vectors can next be used to determine an edge vector. 

The procedure of normalizing the non-defect color plane pixels has been illustrated in 
FIGS. 3a through 3c. 

25 Now, we determine an edge-biased weight Xj that uses the distance between nn = +1 

and nn = -1. A distance value for each vector is determined to be 

5 i = absval{d i [-l]-d i [+l ]}. 
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Then, a measure of an alignment of direction vector i to an edge at a [ mo, n 0 ] is given by: 



x.-u-tf/^sfH/a-i). 

5 where j is an index over all vectors and I is determined by the number of other vectors other 
than the current vector i under consideration. For example, when there are four vectors 
numbered 1 through 4 and the current vector under consideration is i equals 2, then I is 3 (i.e., 
4 minus 1). 

Herein, the parameter k can be adjusted to modify the algorithm's sensitivity to the 
1 0 differences in dj [ nn J , where the sensitivity increases with the value of the exponent k. The 
vector aligned closest to the edge direction (i.e., the minimum value of the absolute 
difference) would have the maximum value of X\. Let this edge vector be symbolized by 
di_4nn]. In FIG. 3c, this step of finding a weighted measure for the edge direction is 
indicated by the arrow. 

15 Finally, the particular pixel under attention a [ mo , n 0 ] is estimated using the weight 

factors Xj to yield the estimation: 

a = S i X i {(d i [-l] + d i [+l ])/2}. 

20 FIGS. 3a - 3c illustrate the various pixel-based algorithm processing steps for 

normalizing. The bar at the right side of the figures relates the various hatching patterns of 
the kernel pixels to a numerical value of the color value which is shown at the right side of the 
bar. FIG. 3a is a Bayer image with its associated edge vectors di , d2 , d3 and 6U in a 7 x 7 
pixel kernel K. A single defect at a[ mo , n 0 ] has been indicated by a white pixel. In FIG. 3b, 

25 the vector pixels near the defect have been shown in an enlarged manner; the ones marked 
with a black dot will not be considered for the defect pixel. In FIG. 3c, the neighboring pixels 
have been normalized to the color plane of the defective pixel. The edge direction is 
evidently vertical and shown in the figure as an arrow, thereby allowing the interpolating of 
the defective point. 
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FIG. 4 illustrates a flow diagram of a preferred algorithm. Here, only the operations 
proper are depicted; commands, handshakes, synchronization and other infrastructure items 
have been considered standard. Now, element 20 is the DSP pipeline of the arrangement 
which is fed by an image sensor not shown and which comprises an appropriate amount of 
5 processing and memory facilities. Now, the DSP pipeline presents an appropriate amount of 
information for being processed according to the present invention. Block 22 represents the 
captured Bayer image data, inasfar as presently used. In block 24, the Bayer image data are 
checked for occurrence of a defect point. If yes, the system goes to the defect correction 
algorithm in block 50. First, in block 26 the data of a 7 x 7 kernel centered at the defect are 

10 constituted. In block 28, the four angle vectors thereof are produced. In block 30, these are 
checked for existence of a column defect. If yes, in block 34, the vertical vector is dropped 
from further consideration. Next, in block 32, the remaining vectors are checked for defects. 
If present, in block 36 these defects are interpolated. Then, either from block 32, or from 
block 36, in block 38 the vector points are normalized to the defect color plane through using 

1 5 gradients. Next, in block 40 a single angular vector is selected, followed in block 42 by linear 
interpolating the defect from the angular vector. Thereupon, block 50 is left with an 
interpolated defect value, that in mixer or accumulator facility 44 is combined with the 
information pertaining to all those situations where no defect point was encountered (block 
24). This terminates the present algorithm. In block 46, other color reconstruction steps and 

20 various further processing may be effected. Finally, the processed information returns to the 
DSP pipeline in block 48. 

Up to now, no relation has been considered between the defect's value and the actual 
scene. The inventor has however recognized that it is possible to check the quality of the 
estimations. In fact, the algorithm is evaluated on the basis of the human visual system and 

25 the Nyquist theorem. Often, a defect value will be acceptable in high-frequency regions 
because the visual system's differentiation ability changes with spatial frequency. Now, the 
above defect correction algorithm attempts to estimate a defective pixel. According to 
Nyquist, the highest non-aliased frequency (the Nyquist frequency f*N) that the Red, Green, 
and Blue pixels may sample is equal to half the maximum frequency they may record in a 

30 particular direction. The highest recordable unalised frequency is at a minimum X A of a cycle 
per pixel (i.e., the full period is 2 pixels long) in a given sample color plane. In a Bayer 



pattern, red for example, is sampled every two pixels, and the Nyquist frequency would be l A 
of a cycle per pixel (i.e., the full period is 4 pixels long). Moreover, the occurrence of a 
defective pixel will further lower the Nyquist frequency by half such that the maximum 
recordable non-aliased frequency would be 1/8 cycle per pixel (i.e., a full period is 8 pixels 
5 long). 

Now, we start by assuming that the CFA (i.e., color filter array) has a minimum 
sampling rate of Va cycle per pixel (i.e., the full period is 4 pixels long). The correlation is 
done through using gradients measured at a two-pixel spacing and so could result in a zero 
gradient through aliasing: then it would be impossible to correlate the non-defect colors. 

10 This could lead to a 100% deviation error. On the other hand, if the defect pixel itself 
deviates only by 10%, using it uncorrected would have been better. Therefore, the typical 
magnitude of a pixel's defect may be used to decide whether the corrected or rather the 
uncorrected value of the pixel is a better approximation. 

When spatial image frequencies surpass the Nyquist frequency, the correction could 

1 5 be unacceptable. In fact, in such high frequency regions many errors would be perceptually 
acceptably anyway. On the other hand, in regions with low spatial frequency the 
requirements set by a user are quite a lot higher; but advantageously, in such a case, the 
algorithm is much less prone to aliasing. Therefore, if the calculated error is greater than the a 
predetermined error, it would be better to only correct for the latter. 

20 FIG. 5 is a diagram showing the reason for aliasing tolerance value t in the correction 

error test. In the figure, the pixels have been shown with their matrix colors Red R and Green 
G, respectively, and their actual values in the vertical direction. The sampled value a for the 
defect pixel is shown by a large "X". As shown, the horizontal arrow 100 has a span of four 
pixel distances. If the limit for the distance from the sampled value a for the defect pixel to 

25 the correction algorithm's estimate a is restricted to the maximum defect deviation €, the 

estimate a would be rejected in favor of keeping the sampled value a. Nevertheless, the 

estimate a is closer to the true but unknown value a' in this example due to the apex (high 

frequency content). Therefore, the tolerance value t, which is a function of the local Nyquist 

frequency, should be added to the maximum defect deviation e to ensure that defect 

30 corrections at apex locations will not be rejected when they would, in effect, be acceptable. 
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FIG. 6 illustrates the creation of limits for the error correction test. The test is 
constructed as follows. Let the sampled value for the defective pixel be a and the true but 
unknown scene value for the defective pixel be a' in an image with an available dynamic 
range C (e.g., such as 255 for an 8-bit image). The algorithm would then produce an estimate 
5 a. In a test, the maximally possible normalized deviation (normalized to the dynamic range) 
for a sampled value from a given defective pixel may have been found, and defined as 

e = abs val { a max - a' } / C 
where a max is the maximum possible deviation that the defective pixel is capable of. 

Rejecting all larger deviations (i.e., larger than eC ) would discard solutions where 

10 the error is less than the error in the value of a(m, n) for certain frequencies. However, in a 
single-defect case, the Nyquist frequency is 1/8 cycle per pixel. In FIG. 3, a Red pixel is 
defective, so that the Red pixels are locally sampled only every four pixels. The maximum 
recordable frequency is 1/4 cycle per pixel, and the Nyquist frequency fN is 1/2 of that. Since 
the adaptive edge algorithm uses gradients to calculate an estimate of the pixel value for the 

15 defect pixel, the maximum non-aliased signal would be 1/8. When the defect pixel occurs at 
a local maximum such as in FIG. 5, the correction will over-estimate the value for the defect 
pixel, although it can never exceed C/16. 

Although the error would not exceed f N C, no system will have a perfect Modulation 
Transfer Function MTF in practice. The MTF may be either theoretically computed or 

20 actually measured. The formal aliasing tolerance value t may therefore be given by t = • 
MTF( f\ ). This tolerance value t is added to the maximum defect deviation value € . Since 
the aliasing tolerance value t is a function of the local Nyquist frequency times the MTF at 
that frequency, the combined maximum defect deviation value and aliasing tolerance value is 
also a function of the local Nyquist frequency. The formal definition of the correction test 

25 may be expressed as follows: 

if abs val { a - a } s ( eC + tC ) 
then, a [ m 0 , n 0 ] = a, 
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otherwise, a [ m 0 , n 0 ] = a. 



FIGS. 6a and 6b illustrate the creation of limits for the error correction test. FIG. 6a 
illustrates the developing of an acceptance range OK for the estimated value a for a sampled 
5 value a of a defect pixel that differs from the true scene value a* by e with an aliasing 
tolerance of t and a scene dynamic range of C. As shown, the estimated value is accepted 
(OK) if the difference with the scene value a' lies between 0 and e C, and the estimated value 
is also accepted (OK) even though the difference is greater than eC, but only if it is greater 
by no more than the aliasing tolerance tC. However, if the error is still greater than the sum 

10 of the maximum defect tolerance £ C and the aliasing tolerance tC, up to possibly 100%, the 
estimated value a. is rejected (NOK). 

FIG. 6b illustrates the range of values for the estimated value a , centered about the 
sampled value a , that can be accepted (OK). Any estimated value a found to deviate from 
the measured sample value a for a defect pixel by less than the maximum defect tolerance 

15 eC added to the aliasing tolerance tC, can be accepted (OK) as a best estimate for the actual 
unknown scene value a*. As shown, the limit Urn can be bidirectional and symmetric, 
provided of course, that the eventual outcome will not be negative. 

FIGS. 7a and 7b illustrate the use of the error correction test for two different 
estimates of a defective pixel. FIG. 7a is an example where estimated value a has an error 

20 greater than (>) a (see est. a err). Therefore, the correction error test rejects a and uses a 
instead to estimate the actual scene value a'. The allowable limits have been indicated +/- 
lim, and the acceptability by OK and NOK, respectively. At the bottom of FIG. 7a, the 
estimated errors err have been indicated. In contradistinction, FIG. 7b is an example where a 
is estimated to have an error smaller than (<) a. Therefore, the correction error test accepts a 

25 for use as an estimate of the actual scene value a'. 

FIG. 8 illustrates a flow diagram of a DSP pipeline provided with an error correction 
test. Much of FIG. 8 corresponds to earlier FIG. 4. In particular, elements 20-24 and 46-50 



correspond to the same numbered elements in FIG. 4 and will not be discussed further. 
However, in block 52 a sampled value a of a defect pixel is retrieve from the captured image 
and provided to both the defect correction algorithm in block 50 (as discussed with respect to 
FIG. 4) and to the correlation test in block 54. The defect correction algorithm in block 50 
5 provides an interpolated defect value as an estimated value a to the correlation test in block 
54. Block 54 execute a correlation test on the two received quantities as discussed above with 
respect to FIGS. 6a, 6b, 7a and 7b. If the interpolated defect value a is accepted, it is 
forwarded to mixer or accumulator block 56 that generally corresponds to block 44 in FIG. 4. 
If the interpolated defect value a. is not accepted, however, block 56 operates as if the original 
1 0 value a was good, or good enough, so that the measured value a is taken as being the best one 
available. 

Having described preferred embodiments of a novel method and apparatus for 
adaptive pixel correction of a multi-color matrix from a sensor (which are intended to be 
illustrative and not limiting), it is noted that modifications and variations can be made by 
15 persons skilled in the art in light of the above teachings. It is therefore to be understood 
that changes may be made in the particular embodiments of the invention disclosed which 
are within the scope and spirit of the invention as defined by the appended claims. 

Having thus described the invention with the details and particularity required by 
the patent laws, what is claimed and desired protected by Letters Patent is set forth in the 
20 appended claims. 
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